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Mechanism for unsupervised clustering 

BACKGROUND OF THE INVENTION 

The invention relates to clustering techniques that are generally 
used to classify input data into groups or clusters without prior knowledge of 

5 those clusters. More particularly, the invention relates to methods and appara- 
tus for automatically determining cluster centres. An example of such cluster- 
ing techniques is a Self-Organizing Map, originally invented by Teuvo Koho- 
nen. The SOM concept is well documented, and a representative example of 
an SOM application is disclosed in US patent 6 260 036. 

10 The current framework under investigation for describing and ana- 

lyzing a context has a critical component based on the clustering of data. This 
clustering is expected to appear at every stage of context computation, from 
the processing of raw input signals to the determination of a higher order con- 
text. Clustering has been well studied over many years and many different ap- 

1 5 proaches to the problem exist. One of the main problems is knowing how many 
clusters exist in the data. Techniques exist to estimate the number of clusters 
in a data set, however the methods either require some form of a priori infor- 
mation or assumptions on the data, or they estimate the number of clusters on 
the basis of an analysis of the' data, which may require storing the data, and be 

20 computationally demanding. None of these approaches seems entirely suitable 
for an on-line, unsupervised cluster analysis in a system with limited resources, 
as would be the case for a context-aware mobile terminal. 

Clustering is an important part of any data analysis or information 
processing problem. The idea is to divide a data set into meaningful subsets so 

25 that points in any subset are closely related to each other and not to points in 
other subsets. The definition of 'related' may be as simple as the distance be- 
tween the points. Many different approaches and techniques can be applied to 
achieve this goal. Each approach has its own assumptions and advantages 
and disadvantages. One of the best-known methods from the partition-based 

30 clustering class is the K-means algorithm, which tries to adaptively position K 
'centres' that minimize the distance between the input data vectors and the 
centres. One of its disadvantages is that the number of the K centres must be 
specified before the clustering is attempted. In the case of an unknown data 
set this may not always be possible. The algorithm can be run several times 

35 with different values of K and the optimum K is chosen on the basis of some 
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criteria. For an on-line system where the data is not stored, this approach is 

slow and impractical. 

Thus a problem associated with the known clustering techniques is 
that while it is relatively easy for humans to determine the cluster centres, such 
5 a determination is difficult for computers. 

BRIEF DESCRIPTION OF THE INVENTION 

An object of the invention is to provide a method and an apparatus 
for implementing the method so as to alleviate the above disadvantages. In 
other words, the object of the invention is to provide a method for automatically 
10 determining cluster centres, such that the method is easily implemented in a 

computer system. 

The object of the invention is achieved by a method and an ar- 
rangement which are characterized by what is stated in the independent 
claims. The preferred embodiments of the invention are disclosed in the de- 

15 pendent claims. 

A computer-implemented method according to the invention can be 

implemented by the following steps: 

initializing a first data structure that comprises a lattice structure of 
weight vectors that create an approximate representation of a plurality of input 
20 data points; 

performing a first iterative process for iteratively updating the weight 
vectors such that they move toward cluster centres; 

performing a second iterative process for iteratively updating a sec- 
ond data structure utilizing the results of the iterative updating of the first data 

25 structure; and 

determining, on the basis of the second data structure, the weight 

vectors that correspond to the cluster centres of the input data points. 

A preferred embodiment of the invention is based on the following 
idea. Self-organizing maps generally use a lattice structure of nodes, and a 

30 weight vector is associated with each node. Each data point in the input data is 
iteratively compared with each weight vector of the lattice, and the node whose 
weight vector best approximates the data point is chosen as the winner for that 
data point and iteration. Then the weight vectors associated with each node of 
the lattice are adjusted. The adjustment made to each node's weight vector is 

35 dependent on the winning node through a neighbourhood function. Following 
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the adjustment of the weight vectors a next iteration step is taken. 

As used in this context, the term 'neighbourhood function" is a func- 
tion of distance on the lattice between the winning node and the node being 
updated such that the value of the function generally decreases as the dis- 
tance increases. With normalized SOMs, the value of the function is one for a 
distance of zero. A common form for the neighbourhood function is Gaussian, 
but preferred embodiments of the invention make use of neighbourhood func- 
tions that are not strictly Gaussian. 

In addition to the primary iteration process for updating the SOM, or 
other clustering mechanism, a second iterative process is run, and the second 
iterative process gives a numerical value for the lattice nodes such that the 
numerical value increases if the node's weight vector is positioned at a cluster 
centre. Then the cluster centres are determined, not on the basis of the weight 
vectors, but on the basis of the numerical values produced by the second itera- 

15 tive process. 

Thus the problem of locating cluster centres reduces to a relatively 

straightforward problem of locating local maxima in the numerical values pro- 
duced by the second iterative process. 

An advantage of the invention is that it is much easier for machines 
to locate local maxima in the numerical values than to locate cluster centres in 
the clustering mechanism wherein the cluster centres are the location in which 
the density of the weight vectors is highest. 

In a preferred embodiment of the invention, the second data struc- 
ture comprises a coefficient for each of the weight vectors in the lattice struc- 
ture. Each iteration in the first iterative process comprises selecting a winner 
weight vector for each of the data points on the basis of a distance measure 
between the input data point and the weight vector. Each iteration in the sec- 
ond iterative process comprises calculating a next value of each coefficient on 
the basis of the current value of the coefficient; and a combination of: 1 ) the 
current coefficient of the winner weight vector, 2) a second neighbourhood 
function that approaches zero as the distance on the lattice structure between 
the weight vector and the winner weight vector increases, and 3) an adjust- 
ment factor for adjusting convergence speed between iterations. 

The combination referred to above can be a simple multiplication. 
If the second neighbourhood function is selected appropriately, such 
that the second data structure has distinct borders, the step of determining the 



20 



30 
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weight vectors can be accomplished simply by selecting local maxima in the 

second data structure. 

A preferred version of the second neighbourhood function is not 
monotonous, but gives negative values at some distances. Also, the second 
5 neighbourhood function is preferably made more pronounced over time as the 

number of prior iterations increases. 

Preferably, the first data structure is or comprises a self-organizing 
map and the input data points represent real-world quantities. 

BRIEF DESCRIPTION OF THE DRAWINGS 
10 in the following the invention will be described in greater detail by 

means of preferred embodiments with reference to the attached drawings, in 
which 

Figure 1 illustrates a self-organizing map (SOM) with six clusters of 
input data points; 

15 Figure 2 shows a typical form of a neighbourhood function used in 

an SOM algorithm; 

Figure 3 shows a 15 by 15 lattice structure resulting from uniformly 

distributed input data; 

Figure 4 shows a probabilistic map for visualizing the cluster centres 

20 in an SOM with six cluster centres; 

Figure 5 shows a preferred form of the second neighbourhood func- 
tion used in the second iterative process according to a preferred embodiment 
of the invention; 

Figure 6 shows a computer pseudocode listing for generating the 

25 function shown in Figure 5. 

Figure 7 shows a coefficient map that visualizes the data structure 

used for locating the cluster centres in the SOM; and 

Figure 8 is a flow chart illustrating a method according to the inven- 
tion wherein the method comprises two iterative processes run in tandem; 
30 Figures 9 and 10 show an SOM map and a coefficient map, respec- 

tively, with five clusters; 

Figures 11 and 12 show an SOM and a coefficient map, respec- 
tively, for an exceptional distribution of input data; 

Figure 13 shows an example of a neighbourhood function used in 

35 an automatic cluster-labelling algorithm; 
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Figure 14 shows the result of the automatic cluster-labelling algo- 
rithm; and 

Figure 15 shows how the automatic cluster-labelling algorithm can 
be integrated with a cluster-determination algorithm according to the invention. 

5 

DETAILED DESCRIPTION OF THE INVENTION 

A practical example of the invention is disclosed in the context of 
self-organizing maps. An SOM is a learning algorithm or mechanism from the 
area of Artificial Neural Networks (ANNs) that find wide application in the area 
10 of vector quantization, unsupervised clustering and supervised classification. 
Reasons for its widespread use include its robustness, even for data sets of 
very different and even unknown origin, as well as the simplicity of its imple- 
mentation. The SOM uses a set of weight vectors to form a quantized, topol- 
ogy-preserving mapping, of the input space. The distribution of the weights 
15 reflects the probability distribution of the input. The SOM representation is 
used in clustering applications generally by clustering the weight vectors after 
training, using for example the K-means algorithm. However the problem of the 
original K-means algorithm still remains, that is, determining the value of K for 
the number of centres. In the following, a method based on the SOM algorithm 
20 is described which can be used to automatically determine cluster centres in 
an unsupervised manner. In other words, the number of clusters does not have 
to be predefined and groups of adjacent SOM weight vectors represent the 
cluster centres. Unlike the K-means algorithm where each cluster is repre- 
sented by one centre, in the inventive algorithm the cluster is represented by a 
25 set of centres which correspond to weight vectors in the SOM. The algorithm 
requires few additional computational resources and makes direct use of the 
information generated during the learning of the SOM. It is already clear why 
the algorithm can be considered a hybrid of the K-means algorithm and a 
method based on a probabilistic mixture model. Each cluster is represented by 
30 a set of centres, which correspond to a set of weights in the SOM. The SOM 
weights, in turn, form an approximation of the probability distribution of the in- 
put. 

From the ANN point of view this may be interesting, as the algorithm 
uses lateral inhibition between the weight vectors to generate the clusters and 
35 a form of Hebbian learning. It is clear that the performance of the clustering 
depends heavily on the topology-preserving and converging ability of the SOM. 
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The SOM algorithm 

Figure 1 shows a self-organizing map 10. More particularly, Figure 1 
shows an online version of SOM. There also exists a "batch SOM" but this re- 
quires storing all the input points and going through all of them several times, 

5 which is why the online-version is preferred here. Reference numerals 1 1 gen- 
erally denote input data points that in most SOM applications represent real- 
world events or quantities. The SOM algorithm creates an SOM or lattice struc- 
ture 12 by means of an iterative process that can be summarized as follows. 
Consider a time sequence of inputs a>(4 t = 1 with co e R m and a probabil- 

0 ity distribution p.. The SOM itself consists of a total of N weight vectors X e R 
distributed on an n-dimensional lattice. Thus there are two associated dimen- 
sions, a dimension m of the input data and the weight vectors, and a dimension 
n of the lattice. The reason for having a lattice is to be able to define 
neighbourhood relations between adjacent weights. For example, if each 

, 5 weight k has an associated position vector i k e r on the lattice, then a distance 
d L (ij, i k ) between weights k andy on the lattice can be defined. The initial val- 
ues of the weight vectors can be randomly chosen, as the convergence of the 
algorithm is independent of the initial conditions. At each iteration, a distance 
d(co(f), X k ) between an input co(0 at time t and each of the weight vectors X k is 

20 calculated. A winner weight v(0 at time t is then defined as: 

v(t) = arg min d(co(t), X k ) I 1 J 

1 SHN 

For real-valued data, a possible distance measure d{ , ) could be the 
Euclidean distance. When the winner has been found, each weight vector is 
25 updated as: 

X,lt+l) - X k (t) + a(0M*, v(0)(co(0 - XAt)) & 

where a(0 e [0,1] is a decreasing function of time which determines 
the learning rate of the SOM and h is a neighbourhood function which is a 
function of the distance on the lattice between the winner weight and the up- 
30 dated weight, that is: h(k, v(*)) = h(d L (i v(t) . i'*)). 

Figure 2 shows a typical form 21 of a neighbourhood function h 
used in the SOM algorithm. The neighbourhood function h is positive, having a 
maximum value of 1 for d L = 0 and decreasing monotonically towards 0 as d L 
increases. In practical situations h is often Gaussian, i.e. of the form: 

35 *<aw-e-i [3] 
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... for some appropriately chosen scaling factor a which may be a 
function of time. The SOM map is built iteratively. In other words, the steps in 
equations [1] and [2] are repeated for each t. For uniformly distributed input 
data and a two-dimensional lattice with 15x15 weights, the SOM algorithm 

5 gives the result shown in Figure 3. 

Figure 3 shows a plot of the weight values on a support [0,1]x[0,1] 
of the input, where adjacent weights on the lattice are connected by a line 
segment in the plot. None of the line segments intersect, which indicates that 
the weights are organized. The weight values are spread uniformly over the 

10 input space and approximate the uniform distribution of the input data. To ar- 
rive at such a result the learning is divided into two phases. In the first phase, 
there is a large a and a, and the algorithm is in the self-organizing phase 
where the weights become organized. The second phase is the convergence 
phase where the neighbourhood width is reduced to zero as a increases and a 

15 approaches zero. The weight vectors converge to the approximation of the 
probability distribution of the input. 

Unsupervised Clustering Based on the SOM 

After the description of the basic SOM algorithm, some techniques 
of determining cluster centres will be disclosed. Consider the SOM shown in 
20 Figure 1 . Figure 1 shows data points generated from six different normal distri- 
butions with different means and variances, and plotted on this there are the 
weight vectors from the SOM when data from this distribution was used as an 
input. It is seen that the weight vectors are organized and are more concen- 
trated at the centres of the clusters. Hence the approximation of the probability 
25 distribution of the input. Reference numerals 13i to 13 6 denote six circles lo- 
cated at the cluster centres. The circles are not normally part of the SOM. 
While it is easy for humans to determine where the cluster centres are, this 
task is surprisingly difficult for computers. 

First, a probabilistic algorithm for determining the cluster centres will 
30 be briefly disclosed. Figure 4 shows a plot of the calculated probability of each 
weight vector being chosen as a winner during the simulation. The probability 
for each weight vector is determined by keeping a record of the number of 
times the weight vector was chosen as a winner and then dividing the number 
by the total number of iterations in the simulation. The i,j axis indicate the po- 
35 sition of each weight on the SOM lattice and the p(i, j) axis shows the probabil- 
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ity of each of the weights. From this probability distribution a cluster structure is 
visible where local maxima in the surface correspond to weight vectors posi- 
tioned at input data cluster centres and local minima form a boundary between 
the local maximum and hence the clusters. However, the different clusters are 
not clearly distinct and the surface is not smooth. This roughness of the sur- 
face and variations in the peak values of the local maxima may lead to prob- 
lems when trying to automatically determine the cluster centres using an algo- 
rithm which would associate a local maximum of the probability with a cluster 
centre. Defining a global threshold above which a weight vector would be con- 
sidered a local maximum and a second global threshold below which a weight 
vector would be considered a local minimum leads to problems. For example, 
if in one part of the lattice the probability at a local maximum is below the 
global threshold to be considered a local maximum hence a cluster centre 
could lead to erroneous choice of cluster centres. In practise, as in the exam- 
ple shown, this is likely to occur. An alternative would be to define local thresh- 
olds for determining a local maximum and a local minimum. However this 
would be a complicated process requiring an involved computation with no 

guarantee of a correct result . 

A clustering algorithm according to a preferred embodiment of the 
invention is based on this observation. In effect the algorithm provides a 
means to smooth the probability surface just described. The use of a 
neighbourhood function means that the smoothing operation is done locally, 
emphasising the local maximum as well as the local minimum. The positive 
elements of the neighbourhood function emphasise the local maximum and the 
negative components emphasise the local minimum. The result is that all the 
local maxima consistently reach high values and the local minima consistently 
reach low values. This allows for the use of a global threshold to identify the 
maximum and minimum and thus facilitates the use of a computer in the proc- 
ess. Hence instead of using directly the probability of a weight being the win- 
ner, a measure somehow related is used in the proposed algorithm which is 

described as follows. 

For each weight i define a scalar coefficient Q. This coefficient is 

bounded to the interval [0, 1], and its initial value before training may be quite 
small. The SOM algorithm is carried out as described earlier. At each iteration 
the winner weight v(t) is determined as in equation [1] and each SOM weight i 
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is updated according to equation [2]. At the same time each coefficient Q is 
updated as: 

C, (t+ 1 ) = Ci(t) + C v(t) (t)h m (d L ) 8 W 
where h m is the second neighbourhood function. d L and v(0 are the 
same terms that were used in the SOM algorithm, that is, d L is the distance on 
the lattice between node i and the winner node v(/). 6 is a small step value for 
adjusting convergence speed. 8 is somewhat analogous to the a in the SOM 
algorithm. 

Following the update Q(J+l) is then forced within the interval [0, 1]. 
For example, if Q(t+1) > 1, it can be set to 1. and if C,(/+l) < 0, it can be set 
to 0.01. Since the update of C,(0 depends on the value of C v(t) , learning is 
clearly Hebbian. 

Figure 5 shows a preferred form 51 of the second neighbourhood 
function h m . Actually, Figure 5 shows a time-dependent version of the second 
neighbourhood function h m where the function becomes more pronounced as 
the number of prior iterations increases. In other words, with increasing time, 
the function h m achieves negative values sooner (as the distance d L in- 
creases), and the negative values are much more negative than during the 
earlier iterations. 

As can be seen, the preferred form 51 of the second neighbourhood 
function h m somewhat resembles the first neighbourhood function h used in the 
SOM algorithm. Like the first neighbourhood function h, the second neighbour- 
hood function h m starts at 1 when the distance is zero. Also, h and h m both ap- 
proach zero as the distance d L increases. 

However, for some distances, the second neighbourhood function 
h m is preferably negative. For instance, in a 10 by 10 lattice. h m may be nega- 
tive for distances over 3. The negative value of the second neighbourhood 
function h m can be seen as a form of lateral inhibition between the weights. 
Lateral inhibition is a mathematical model that tries to approximate real biologi- 
30 cal phenomena. Similar to the h function used in the SOM, weights adjacent to 
the centre of activity have their coefficients and hence their activity increased, 
while the activity of weights further away from the centre of activity are inhib- 
ited. 

This lateral inhibition is rarely if ever used in practical applications. 
35 however. In the SOM, the interaction between the weight vectors is defined by 
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the neighbourhood function h defined by equation [3], which is strictly positive. 
If h was allowed to be negative at any point, divergence of the weight vectors 
could result, instead of convergence. In the clustering method proposed here 
this lateral inhibition is used to determine the cluster centres. 
5 Intuitively it can be seen that if weight i is quite often the winner then 

C will increase along with its neighbours. Similarly, when the winner is U for its 
closest neighbours; at a small distance from i on the lattice such that that h m is 
positive, the q will also increase. At the same time, fory at a large distance 
from i on the lattice, where h m is negative, the Q will decrease. Similarly, if i is 
o not often the winner, its C, will not increase very much and will be decreased 
by other winners located at a distance on the lattice. Given the example .n Fig- 
ure 1 it is clear that weights in the inter-cluster regions will have a lower prob- 
ability of being winners, whereas weights close to the cluster centres will have 
a higher probability of being winners. Hence it would be expected that the coef- 
15 ficients C, would be higher for weights positioned in or near the cluster centres 
and small for positions between the clusters. This would then provide bounda- 
ries between the cluster centres. Of course this depends on the fact that the 
weight vectors X t reach an organized configuration. 

Figure 6 shows an example of a computer pseudocode listing for 

20 generating the function shown in Figure 5. 

Figure 7 shows a plot 70 of the coefficient values C(i, j) for the 
weights i, j in the SOM of Figure 1 with 6 = 0.01 after 20000 iterations. The 
second neighbourhood function h m varied with time and initially did not have a 
high level of lateral inhibition. Towards the end of the simulation, the level of 
25 lateral inhibition was increased. The surface shown in Figure 7 has six distinct 
and separated elevated regions. As a result of forcing the Q between 0 and 1, 
each elevated region corresponds to a set of adjacent weights on the lattice 
whose coefficients Q have saturated at or near 1 and whose weights X t are 
found at the cluster centre. These regions are surrounded by regions where 
30 the coefficients Q have been driven to small values. Hence it is poss.ble to 
determine which weights represent the same cluster. To determine which clus- 
ter an input vector belongs to. simply find the closest centre and assign the 
input to the cluster to which the centre belongs. Thus unlike the K-means algo- 
rithm where one weight represents the cluster, in the algorithm proposed here 
35 a group of weights represents the cluster. The means of classification .s the 
same. Because the coefficients Q are saturated near 0 or 1 . it is a simple task 
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to determine the cluster centres using a global threshold, the value of which 

could be set for example at 0.5. 

It should be noted that the plot 70 is for visualization purposes only 
and is not required by computers. Instead, reference number 71 points to an 
5 array of current coefficients Q(t) and reference number 72 to an array of next 
coefficients C t {t+l). It is the array 72 of updated coefficients that a computer 
uses to determine the cluster centres and their locations. An arrow 806 de- 
notes updating of the coefficients that takes place in step 806 of Figure 8 that 
will be described next. 
10 Figure 8 is a flow chart illustrating a method according to a preferred 

embodiment of the invention wherein the method comprises two iterative proc- 
esses 81 and 82 run in tandem. The odd-numbered steps on the left-hand side 
of Figure 8 relate to the known SOM algorithm. The even-numbered steps on 
the right-hand side of Figure 8 relate to the inventive algorithm for maintaining 
15 and updating the second data structure that is used to determine the cluster 
centres automatically. In step 801 the SOM algorithm is initialized. The initiali- 
zation comprises selecting initial values for the a, a, h and randomly initializing 
the weight vectors X, Since Figure 8 shows an online algorithm, the values of 
the inputs <o(t) are not known at this stage. Step 802 is a corresponding ini- 
20 tialization step for the second data structure. Steps 803. 805. 807 and 809 
form the conventional SOM iteration. In step 803. input co{t) is presented to 
the SOM at iteration t. In step 805. the winner weight v(t) is selected according 
to equation [1]. In step 807. the weight vectors are updated according to equa- 
tion [2]. In an optional step 807'. the variables a that determine learning speed 
25 and/or the a of the first neighbourhood function h are updated. In step 809. the 
iterative loop is repeated until some predetermined stopping criteria are met. 
For instance, the loop may be set to run a predetermined number of times, or 
the loop may be interrupted when each succeeding iteration fails to produce a 
change that exceeds a given threshold. 
30 Steps 806 and 808 relate to the second iterative process 82 for 

maintaining and updating the second data structure that is used to determine 
the cluster centres. In step 806. the coefficients C, are updated on the basis of 
the winner weights v(t) according to equation [4]. In an optional step 808. pa- 
rameters for the second neighbourhood function h m are updated (see Figure 
35 5). According to a preferred feature of the invention, steps 806 and 808 of the 
second iterative process 82 are interleaved with the steps of the first iterative 
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process 81. In this way, step 806 utilizes intermediate calculation results of 
step 805. Similarly, step 808, in which the second neighbourhood function h m is 
updated, may utilize data from step 807 that updates the variable a and the 

first neighbourhood function. 

Figures 9 and 1 0 show an SOM map and a coefficient map, respec- 
tively. In this example the number of clusters in the input data distribution was 
reduced by one and the same SOM algorithm was used. Figure 9 shows the 
result of both the input and the final configuration of the weight vectors. Figure 
10 shows the resulting C, values. The five cluster centres are once again quite 
clear. It is remarkable that the same algorithm, without any adjustments, was 
capable of finding the new number of clusters and the locations of those clus- 
ters. , . 

The way the invention works is as follows. In the beginning there is 

a predefined lattice, which in this case is two-dimensional. Each point of the 
15 lattice is given a label, e.g. (2,3). (0,15). This lattice remains fixed and the la- 
belling of the lattice points does not change. In the above examples, the lattice 
structure is a 1 5x1 5 lattice. It is from the lattice that the distances dL used in all 
neighbourhood functions are determined. For instance, the distance between 
lattice points (1,3) and (7,8) could be 6. depending on the distance measure 
20 we use. 

Each lattice point is associated with a weight vector. The dimension 
of the weight vector is always the same as the dimension of the input data vec- 
tor In the examples here the input data has two dimensions. It is the weight 
vectors that change depending on the input data point and the distance be- 
25 tween two points on the lattice, the first point being the lattice point associated 
with the winner and the second point the lattice point associated w.th the 
weight vector to be updated. This distance is not used to update the weight 
vector directly, but to determine the value of the first neighbourhood function in 

the update of the weight vector. 

Figures 1 and 7 show a two-dimensional plot of the input data points 
and the weight vectors from the SOM. The plot is in the input space and by 
chance all the input points were bounded by [-8, 8]. The data points are just 
the points shown. The weight vectors are plotted at the crossing point of the 
lines. Another way of looking at this is to draw a line between weight vectors 
whose associated lattice nodes are the closest nodes on the lattice. The fact 
that the plot appears as a lattice means that the weights are organized, that is, 
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the weight vectors associated with adjacent lattice nodes appear in the input 
data space as adjacent to each other. 

The relationship between the coefficients and the SOM lattice is the 
same as the relationship between the weight vectors and the SOM lattice, ex- 
5 cept that the dimension of the coefficients is always 1. The relationship is 
somewhat similar, though not the same as a probability measure, where the 
probability would be that the weight vector associated with the lattice node with 
which the coefficient is associated will be chosen as the winner for any given 
data input. Another interpretation is that the coefficients somehow represent an 
10 exaggerated version of the probability distribution of the input data. 

In conclusion, we might say that the lattice is a fixed structure. 
There is one weight vector associated with each lattice node. The weight vec- 
tor is in the input data space. Similarly, there is one coefficient associated with 
each lattice node. It is a scalar value and represents an indication of probabil- 
15 ity, though not the real probability that the weight vector associated with the 
same lattice node will be chosen as winner for a given input data point. 

The dimensionality of the input data and the lattice are not neces- 
sarily the same. The input data may have any number of dimensions, such as 
5, 10 or 15. In that case the dimensions of the weight vectors would also be 5, 
20 10, or 15. But the lattice could still be two-dimensional. We could also choose 
a different lattice to begin with (i.e. change the SOM structure) and make it 
four-dimensional, for example. In this case, if we still choose to have 15 lattice 
nodes along each axis of the lattice then we would have 15x15x15x15 lattice 
nodes and associated with each lattice node, a 5. 10 or 15-dimensional weight 
25 vector. The examples above use a two-dimensional lattice and a two- 
dimensional input space merely because it is easier to draw and visualize. In 
practical implementations one could expect that the input data has more di- 
mensions but the lattice structure could be two-dimensional. The number of 
lattice nodes along each dimension of the lattice is variable depending on the 
30 amount of computational resources available. 

Figure 1 1 shows a mixed clustering example consisting of two very 
different clusters, namely a first cluster 112 described by a normal distribution 
and a second cluster 114 described by a uniform distribution along a parabola. 
The same SOM was used as in the previous two examples. Figure 12 shows a 
35 map 120 of coefficients C u for this example. There are two distinct areas. 
There is a connected region 122 of high values around three edges of the lat- 



WO 03/075221 





CT7FI03/00152 



14 

tice which correspond to the parabolic distribution cluster 1 1 2 in Figure 1 1 , and 
the disc shape 124 of connected values which corresponds to the normal dis- 
tribution 1 14. This example is quite interesting because of the complexity of the 
parabolic distribution. In the mixture model clustering algorithm, without any 

5 prior information it would be difficult to generalize this distribution to a normal 
distribution. Similarly, a K-means approximation would have difficulty in resolv- 
ing these two clusters as at some points the distance between two points in 
different clusters is smaller than the distance between two points in the same 
cluster. In experiments on this example the best K-means came up with four 

10 clusters. This example indicates that the clustering algorithm according to the 
invention may be very general. 

Automatic labelling of cluster centres 

A further preferred embodiment of the invention relates to automatic 
and unsupervised labelling of the clusters. The same notation is used here as 

15 above and only notation pertinent to this embodiment will be explained. Con- 
sider a set of labels B = {1, 2, ... , which will be used to label the clusters. 
In practise K should be at least greater than or equal to the expected number 
of clusters. In the case of no prior knowledge, it may be suitable to let K - N, 
the total number of weights in the SOM, as this imposes a limit on the maxi- 

20 mum number of clusters which can be identified. 

For each weight i in the SOM, define a vector of coefficients 0, as: 

ei-(6U*U- fa) [5] 

Each coefficient 0 U e [0, 1] represents a weighting between the 
SOM node i and the label /. The weight i belongs to cluster / if: 

25 / = arg max 6 lM ™ 

1 £k<K 

The updating algorithm used on these coefficients to achieve auto- 
matic labelling proceeds as follows. At time t SOM weight v(f) is chosen as the 
winner. The weight and its neighbours are updated as in the normal SOM algo- 
30 rithm Also the coefficient Q is updated as well as the coefficients Q of the 
neighbours of Q. The updating of the coefficients Q and the interpretation of 
the results form the basis of the main invention, namely the automatic and un- 
supervised clustering. 
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In the automatic and unsupervised labelling of the clusters, at the 
same time t the <S* are updated as follows. Define lv(t) as the label of the clus- 
ter to which the winner weight v(0 is assigned, thus from equation (6). 

/v(0 = arg max 6 vm {t) ™ 
5 i<k£K 

For all the weights j,j = \ NXhe components 0 JMl) are then up- 
dated as follows: 

0jMO (' + D = 0 JMO W+ C v(l) (t+l) h£d L )S [8] 
where once again h B \s a neighbourhood function and preferably has 
10 the form shown in Figure 13. The idea is that the neighbours of the winning 
weight will have their 9 m) increased to ensure that they will be classified to 
the same cluster as the winner v(/). whereas weights further away from the 
winner weight will have their 0 jM0 decreased to ensure that they will be class.- 
fied to a different cluster than the winner. 
15 For the weights j where the neighbourhood function h^d L ) > 0 it is 

also advantageous to decrease the other coefficients B jMth k=l,...K,k* lv(t) 
in &j as follows: 

0j.k(t+\) = 6j, k {i) - C v(t) {t+\)h B {d L )S [9] 
This reinforces the labelling of the winner and its neighbours to the 

20 cluster label /v(r). 

Note that equation [4] uses C v at iteration t whereas equations [8] 

and [9] use C v at iteration f+1. Actually, the C v coefficient changes so little be- 
tween iterations that either value can be used, depending on which value is 

more conveniently available. 
25 Figure 14 shows an example of the result of the combination of the 

SOM automatic clustering and automatic labelling of the clusters in the case of 
an input distribution consisting of five normal distributions. In this case the set 
of labels was given by L = {1, 2, 3, 4, 5, 6, 7, 8}. The values of the ^ were 
randomly initialized. Figure 14 shows an SOM with five distinct regions, one 

30 region around each cluster. Each node in the five areas 1 to 5 is ass.gned a 
label such that nodes in area 1 have a label of 2, nodes in area 2 have a label 
of 4, etc. The inter-region areas have a label of 0. These weights had a maxi- 
mum of 0, lower than a threshold value of 0.2 and therefore they are not as- 
signed as centres to any cluster. It is clear that the cluster centres have been 

35 properly labelled with the labels {2, 4, 5, 6, 8} . 
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Figure 1 5 shows how the automatic cluster-labelling algorithm can 
be integrated with the cluster-determination algorithm according to the inven- 
tion Figure 15 is a modification of Figure 8. with step 806 followed by step 806' 
that relates to the automatic cluster-labelling algorithm. In step 806, the cluster 
labels lv(t) are determined according to equation [7]. Then the 0 JMO compo- 
nents are updated according to equation [8] and the 9 jM components are up- 
dated according to equation [9]. By placing step 806' inside the second itera- 
tive process 82, maximal synergy benefits are obtained, or in other words, the 
computational overhead is kept to a minimum because step 806' makes used 
of the winner selection and coefficient determination already performed for the 
SOM construction and the automatic cluster determination. 
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Summary 

The technique according to the invention allows automatic determi- 
nation of cluster centres with a minimal amount of information on the data. No 
explicit, initial estimate of the number of clusters is required. Given the nature 
of convergence of the SOM, there is no need to know the type of distribution of 
the clusters either. In this respect the algorithm is very general. However, al- 
though explicit initial estimates of the number of clusters are not required, care 
should be taken to ensure that the SOM lattice contains a number of nodes 
larger than the expected number of clusters, as well as choosing a non- 
monotonous neighbourhood function that is negative for large distances and 
provides a level of lateral inhibition to ensure that the coefficients for the cluster 

regions stand out more clearly. 

The preferred embodiment of the invention, in which the second it- 
erative process is interleaved with the conventional iterative process, requires 
little computational overhead. Thus this embodiment of the invention is espe- 
cially suitable for on-line application where human supervision is not available. 
Initial simulations on artificial data show that the inventive technique is simple 
and apparently robust and is more easily generalized than most current clus- 
tering algorithms. The technique according to the invention can be considered 
somewhat as a hybrid of the K-means and probabilistic-model-based cluster- 
ing- 
It is readily apparent to a person skilled in the art that, as the tech- 
nology advances, the inventive concept can be implemented in various ways. 
35 The invention and its embodiments are not limited to the examples described 
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above but may vary within the scope of the claims. 
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CLAIMS 

1 . A computer-implemented method for determining cluster centres 
(13i - 13e) in a first data structure (10), wherein the first data structure com- 
prises a lattice structure (12) of weight vectors that create an approximate rep- 
5 resentation of a plu rality of input data points (11); 

the method comprising: 

performing a first iterative process (81) for iteratively updating the 
weight vectors such that they move toward cluster centres (13i - 13 6 ); 

performing a second iterative process (82) for iteratively updating a 
10 second data structure (70 - 72) utilizing results of the iterative updating of the 

first data structure; and 

determining, on the basis of the second data structure (70 - 72), the 

weight vectors that correspond to cluster centres of the input data points. 

2. A method according to claim 1 , wherein each iteration in the first 

1 5 iterative process (81 ) comprises: 

selecting a winner weight vector (v) for each data point on the basis 

of the distance between the data point and the weight vectors; and 

calculating a next value for each weight vector on the basis of the 
current value of the weight vector and a first neighbourhood function (21 , h) of 
20 the distance on the lattice structure between the weight vector and the winner 

weight vector; and 

the second data structure (70 - 72) comprises a first coefficient (G) 

for each of the weight vectors in the lattice structure and each iteration in the 

second iterative process (82) comprises calculating (806) a next value of each 

25 first coefficient (Of) on the basis of: 

the current value of the first coefficient; and a combination of: 
a first coefficient of the winner weight vector (v), 
a second neighbourhood function (51 , h m ) of the distance on the lat- 
tice structure between the weight vector and the winner weight vector, and 

30 an adjustment factor (8) for adjusting convergence speed between 

iterations. 

3. A method according to claim 1 or 2. wherein the step of determin- 
ing the weight vectors that correspond to cluster centres comprises selecting 
local maxima in the second data structure (70 - 72). 
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4. A method according to claim 2 or 3, wherein the combination is or 
comprises multiplication. 

5. A method according to any one of claims 2 to 4, wherein the sec- 
ond neighbourhood function (51 . h m ) is not monotonous. 

6. A method according to any one of claims 2 to 5, wherein the first 
coefficients are limited to the range [0,1] and the second neighbourhood func- 
tion (51, h m ) gives negative or positive values, respectively, for some dis- 
tances. 

7. A method according to any one of claims 2 to 6, wherein the sec- 
ond neighbourhood function (51 , h m ) depends on the number of prior iterations. 

8. A method according to any one of the preceding claims, wherein 
the input data points (1 1 ) represent real-world quantities. 

9. A method according to any one of claims 2 to 8, wherein the first 
data structure (10) is or comprises a self-organizing map. 

« 

10. A method according to claim 9, further comprising: 
estimating an upper limit K for the number of clusters in the self- 
organizing map; . 

defining a coefficient vector ®i = {0 U , 0 U2 , ... 0 iiK ) for each weight 
vector i in the self-organizing map. the coefficient vector comprising K second 
coefficients each of which represents a weighting between the weight vec- 
tor i and a label /; and 

assigning cluster label / to weight vector i if: 

/ = arg max 0 i k . 

1 £k<K 

11. A method according to claim 10, wherein each iteration in the 
second iterative process (82) comprises calculating (806 1 ) a next value of each 
second coefficient on the basis of the current value of the second coefficient 

and a combination of: 

a coefficient of the winner weight vector, 
30 a third neighbourhood function (1 31 , h B ) of distance, and 

an adjustment factor (8) for adjusting convergence speed between 

iterations. 
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12. A computer-readable program product comprising a computer 
program code, wherein executing the computer program code in a computer 
causes the computer to carry out the steps of the method according to claim 1 . 
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x=linspace(0,10,100); 
figure 

for i=1:1.5:10 

p=0.3+(0.4*(1-(i/10))); 
y=exp(-0.03*x.*x*i).*(p-(0.ori*x*x))/p; 

plot(x,y) 
hold on 

end 
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Initialize first data structure, 
parameters a, a, h, Xj, i s 1, ...,N 
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Present input a>(t) to the SOM 



On the basis of the input, determine 
winner node v(t) using eq. [1] 
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Update weight vectors Xj(t) [eq. 2] 



Optionally, update a, a,h 
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Finish if stopping conditions reached 
otherwise t=t+1 , perform next iteration 



Initialize second data structure, 
parameters d, h m , Cj, i = 1 N 
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On the basis of the winner weights, 
update coefficients Cj [eq. 4] 



Optionally, update 
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On the basis of the winner weights, 
update coefficient s C; [eq. 4] 

1. determine cluster labels lv(t) [eq. 7] 

2. update components 9 Jt i v ( t ) [eq. 8] 

3. update components d jt k(t) [eq- 9] 
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cited to establish the publication date of another atabon or other 
special reason (as specified) 

"O" document referring to an oral disclosure, use, exhibition or other 
means 

•P* document published prior to the international filing date but later than 

the priority date cl aimed — — 

Date of the actual completion of the international search 



99 Mav 2003 

Name and mailing address of the ISA/ 

Swedish Patent Office 
Box 5055, S-102 42 STOCKHOLM 
Ferrule No. + 46 8 666 02 86 



See patent family annex. 

T* later document published after the international filing date or priority 
date and not in conflict with the application but cited to understand 
the principle ot theory underlying the invention 

-X* document of particular relevance: the claimed invention cannot be 
considered novel or cannot be considered to involve an inventive 
step when the document is taken alone 

•Y- document of particular relevance: the Maimed invention cannot be 
considered to involve an inventive step when the documents 
combined with one or more other such documents, such combination 
being obvious to a person skilled in the art 

*&* document member of the same patent family 



Authorized officer 

Jenny Forss /LR 

Telephone No- +46 8 782 25 00 



Form PCT /ISA/210 (second sheet) (July 1998) 



INTERNATIONAL S&4RCH REPORT 



Internatioi 

PCT/FI 



(plication No. 
00152 



C (Continuation). DOCUMENTS CONSIDERED TO BE RELEVANT 
Category* 1 Citation of document, with indication, where appropriate, of the relevant passages 



BEZDEK, J.C. et al.: Fuzzy Kohonen clustering 

networks. IEEE International Conference on Fuzzy 
systems (Cat.no.92CH3073-4), San Diego, CA, USA, 
8-12 March 1992. New York 1992. ISBN 0-7083-0236-2, 
pages 1035-1043 



LEE, H.S. et al.: An investigation into 

unsupervised clustering technoques Proceedings of 
the IEEE SoutheastCon 2000. 'Preparing for the New 
Millennium' (cat.no.00CH37105) , Nashville, USA 
April 2000, ISBN 0-7803-6312-4, pages 124-130 



Relevant to claim No. 
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VESANTO, 



niu, J. et al.: Clustering of the 
Self -Organizing Map. IEEE Trans. Neural Netw. (USA) 
May 2000, IEEE, USA. ISSN 1045-9227, vol.11 no. 3, 
pages 586-600 
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From the INTERN ATI ON AI^UREAU 



PCT 



NOTICE INFORMING THE APPLICANT OF THE 
COMMUNICATION OF THE INTERNATIONAL 
APPLICATION TO THE DESIGNATED OFFICES 

(PCT Rule 47. 1(c), first sentence) 



Date of ma\\\ng( day/montWyear) 

12 September 2003 (12.09.03) 



Applicant's or agent's file reference 

2020342PC/ko 

International application No. 

PCT/FI03/00152 

Applicant 



To: 



m 



KOLSTER OY AB 
Iso Roobertinkatu 23 
P.O.Box 148 
FIN-00121 Helsinki 
FINLANDE 



IMPORTANT NOTICE 



International filing dale(day/mont}t/year) 

03 March 2003 (03.03.03) 



NOKIA CORPORATION 



Priority date( dav/montWyear) 

04 March 2002 (04.03.02) 



Notice is hereby given that the International Bureau has communicated, as provided in Article 20, the international application to the 
following designated Offices on the date indicated above as the date of mailing of this notice: 



AU, AZ, BY, CH, CN, CO, DE, DZ, HU, JP, KG, KP, KR, MD, MK, MZ, RU, TM, US 



^ d„i„ Ai i (r\ third sentence those Offices will accept the present notice as conclusive evidence that the communication 
of SSSSS ISSSiS S.™nVace o„°;L date of maiLg indicated above and no copy of the .taxational applicauon is 
required to be furnished by the applicant to the designated Office(s). 



2. The following designated Offices have waived the requirement for such a communication at this time: 



Ic/sD SE, SG SKSL if™ TR, XT, TZ, UA, UG, UZ, VC, VN. YU. ZA, ZM, ZW 
The communication will be made to those Offices only upon their request. Furthermore, those Offices do not require the applicant to 
furnish a copy of the international application (Rule 49. l(a-fcw)). 

Enclosed with this notice is a copy of the international application as published by the International Bureau on 
1 2 September 2003 (1 2.09.03) under No. 03/075221 

TIME LIMITS for filing a demand for international preliminary examination and for entry into the national phase 

The applicable time limit for entering the national phase will, subject to what is said in the following paragraph be 30 MONTHS from 
The applica ble time nmnv or en i fe y demand for international preliminary examination is filed before the 

S^SflSnE^ the 'priority datl but aTso in respect of any designated Office, in the absence of filing of such demand where 
S-a as 3 fied with effect from I April 2002 applies in respect of that designated Office. For further details, see PCT Gazette 
^ 44/2001 of " November 2001, pages 19926, 19932 and 19934, as well as the PCT Newlener, October and November 2001 and 
February 2002 issues. 

m notice time limits other than the 30-month time limit will continue to apply, for various periods of lime, in respect of certain 
WIPO's Internet site, at http://www.wipo.int/pcl/en/index.html. 

For filinc a demand for international preliminary examination, see the PCT Applicant's Guide, Volume UA, Chapter IX. Only an 
Z^lST^nA or resident of a PCT Contracting State which is bound by ^Chapter II has the right to file a demand for 
international preliminary examination (at present, all PCT Contracting States are bound by Chapter II). 
It is the applicant's sole responsibility to monitor all these time limits. 



The International Bureau of WIPO 
34, chemin des Colombettes 
12 1 1 Geneva 20, Switzerland 



Facsimile No.(41-22) 740.14.35 
Form PCTAB/308 (April 2002) 



Authorized officer 



Judith Zahra 



Telephone No.(41-22) 338.91.11 




PATENT COOPERATION TREATY 
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INFORMATION CONCERNING ELECTED 
OFFICES NOTIFIED OF THEIR ELECTION 

(PCT Rule 61 .3) 



Date of mailing (day/month/year) 

08 October 2003 (08.10.03) 



From the INTERNATIO 

To: 



BUREAU 




Applicant's or agent" s file reference 
2020342PC/ko 

International application No. 
PCT/FI03/00152 



IMPORTANT INFORMATION 



International filing date (day/month/year) 

03 March 2003 (03.03.03) 



Priority date (day/month/year) 

04 March 2002 (04.03.02) 



Applicant 



NOKIA CORPORATION et al 



1 The applicant is nere.y informed tnat Z ~, Bureau nas. according to Artic.e 31 (7), notified eacn of tne fol.owing 
• offices £~CH,CY,C^ ' 

Sal :BG,CA,CN,DE,GB,IL^P,KP,KR,MN,NO,PURO,RU,SK,US 

2 THefoUowin.OfficesHavewaivedtnereouirementfortHenotificatio^ 

2 - J the International Bureau only upon the.r request: 

APGH,GM,KE,LS,MW,MZ,SD,SL,SZ,TZ,UG,ZM,ZW 

EA AM AZ,BY,KG,KZ,MD,RU,TJ,TM 

OA :^F^BJ,CF,CG,CI,CM,GA,GN,GQ,GW,MU CU,CZ,DK,DM,DZ,EC, 
National :AE,AG,AL,AM,AT,AU AZ BA BB BR ^BY,BZ CH CO^k,^u ^ ^ 

EE,ES,FI,GD,GE,GH,GM,HR,HU ID^N^ 

MG,MK,MW,MX,MZ,NZ,OM,PH,PT,SC,SD,SE,SG,SL f TJ,TM,liM,irt,i , 
VN,YU,ZA,ZM,ZW . 

of the PCT Applicant's Guide, 
the purposes of obtaining a European patent. 



The International Bureau of WIPO 
34, chemin des Colombettes 
121 1 Geneva 20, Switzerland 



Facsimile No. (41 -22) 338.87.20 



Authorized officer: 



El Mostafa MOUSSAID (Fax 338-87-20) 



Telephone No. (41-22) 338 9242 



5899566 



Form PCT/1B/332 (September 1997) 



•v 



t f/ 





1/2 



HEP 2DK 



10/506634 



2020342PC/ko 



Original (for SUBMISSION) - printed on 04.03.2003 02:25:00 PM 



Vlll-4-1 I Declaration: Inventorship (only for 
the purposes of the designation of 
the United States of America) 

Declaration of inventorship (Rules'* 
4.17(iv) and 51bis.1(a)(iv)) for the 
purposes of the designation of the 
United States of America: 




VI 1 1-4-1 Prior applications: 
-1 



I hereby declare that I believe I am the 
original, first and sole (if only one 
inventor is listed below) or joint (xf 
more than one inventor is listed below) 
inventor of the subject matter which is 
claimed and for which a patent is 
sought . 

This declaration is directed to 
international application No. 
PCT/FI03/00152 (if furnishing declaration 
pursuant to Rule 2 6ter) 
I hereby declare that my residence, 
mailing address, and citizenship are as 
stated next to my name. 

I hereby state that I have reviewed and 
understand the contents of the 
above- identified international 
application, including the claims of 
said application. I have identified in 
the request of said application, ,in 
compliance with PCT Rule 4.10, any claim- 
to foreign priority, and I have 
identified below, under the heading 
"Prior Applications," by application 
number, country or Member of the World 
Trade Organization, day, month and year 
of filing, any application for a patent 
or inventor's certificate filed in a 
country other than the United States of 
America, including any PCT international 
application designating at least one 
country other than the United States of 
America, having a filing date before 
that of the application on which foreign 

priority is claimed. 

20020414, FI, 04 March 2002 (04.03.2002) 
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I hereby acKnowxeage the duty to 
disclose information that is known by me 
to be material to patentability as 
defined by 37 C.F.R. § 1.56/ inclu 
for continuation-in-part applications, 
material information which became 
available between the filing date of the 
prior application and the PCT 
international filing date of the 
continuation-in-part application. 
I hereby declare that all statements 
made herein of my own knowledge are true 
and that all statements made on 
information and belief are believed to 
be true; and further that these 
statements were made with the knowledge 
that willful false statements and the 
like so made are punishable by fine or 
imprisonment, or both, under Section 
1001 of Title 18 of the United States 
Code and that such willful false 
statements may jeopardize the validity 
of the application or any patent issued 



Vlll-4-1 
-1-6 



Name: 
Residence: 

(city and either US State, if applicable, 
or country) 
Mailing address: 

Citizenship: 

inventor's Signature: 
(if not contained in the request, or if 
declaration is corrected or added under 
Rule 26ter after the filing of the 
international application. The signature 
must be that of the inventor, not that of 
the agent) 
Date: 

(of signature which is not contained in 
the request, or of the declaration that is 
corrected or added under Rule 26ter 
after the filing of the international 
application 



FLANAGAN, Adrian 
Helsinki, Finland 




Vallilantie 36 A / 4 
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For receiving Office use only 

International Application No. 

International Filing Date 



Name of receiving Office and "PCT 
Internationa! Application" 



PCT/FI03/08152 

0 3 MAR 2003 (03.0103) 

The Finnish Patent Office 
PCT International Application 



Form - PCT/RO/101 PCT Request 

repared using 



Petition 

The undersigned requests that the 
present international application be 
Drocessed according to the Patent 
Cooperation Treaty 



PCT-EASY Version 2.92 
(updated 01.01.2003) 



Receiving Office (specified by the 
applicant) 



National Board of Patents and 
Registration (Finland) (RO/FI) 



Applicant's or agent's file reference 
Title of invention 



Applicant 

This person is: 
Applicant for 
Name 
Address: 



State of nationality 
State of residence 
Applicant and/or inventor 

This person is: 
Applicant for 
Name (LAST, First) 
Address: 



State of nationality 
State of residence 



2020342PC/ko 

MECHANISM FOR U NSUPERVISED CL USTERING 

applicant only 

all designated States except US 
NOKIA CORPORATION 
Keilalahdentie 4 
FIN- 02150 Espoo 
Finland 
FI 
FI 

applicant and inventor 
US only 

FLANAGAN, Adrian 
Vallilantie 36 AQQ^H* 
FIN-00510 Helsinki 
Finland 

FI 



PCT REQUEST 



PCT/Fl 0 3 / 0 0 1 5 2 
' # 2/5 # 
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IV-1 



IV-1-1 
IV-1 -2 



V-2 



Agent or common representative; or 
address for correspondence 

The person identified below is 
hereby/has been appointed to act on 
behalf of the applicant(s) before the 
competent International Authorities as: 

Name 
Address: 



Telephone No. 
Facsimile No. 



Designation of States 



Regional Patent 

(other kinds of protection or treatment, 
if any, are specified between 
parentheses after the designation (s) 
concerned) 



National Patent 
(other kinds of protection or treatment, 
if any, are specified between 
parentheses after the designation (s) 
concerned) 



KOLSTER OY 
Iso Roobertinkatu 23 
P.O.Box 148 
FIN-00121 Helsinki 



358 9 618 821 
+358 9 602 244 

AP: GH GM KE LS MW MZ SD SL SZ TZ UG ZM 
ZW and any other State which is a 
Contracting State of the Harare Protocol 
and of the PCT 

EA: AM AZ BY KG KZ MD RU TJ TM and any 
other State which is a Contracting State 
of the Eurasian Patent Convention and of 
the PCT 

EP: AT BE BG CH&LI CY CZ DE DK EE ES FI 
FR GB GR HU IE IT LU MC NL PT SE SI SK 
TR and any other State which is a 
Contracting State of the European Patent 
Convention and of the PCT 

OA: BF BJ CF CG CI CM GA GN GQ GW ML MR 
NE SN TD TG and any other State which is 
a member State of OAPI and a Contracting 
State of the PCT 



AE AG AL AM AT (patent and utility 
model) AU AZ BA BB BG BR BY BZ CA CH&LI 
CN CO CR CU CZ (patent and utility 
model) DE (patent and utility model) DK 
(patent and utility model) DM DZ EC EE 
(patent and utility model) ES FI (patent 
and utility model) GB GD GE GH GM HR HU 
ID IL IN IS JP KE KG KP KR KZ LC LK LR 
LS LT LU LV MA MD MG MK MN MW MX MZ NO 

NZ OM PH PL PT RO RU SC SD SE SG SK 
(patent -and utility model) SL TJ TM TN 
TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW 
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■y^g [ precautionary Designation Statement 

In addition to the designations made 
under items V-1, V-2 and V-3, the 
applicant also makes under Rule 4.9(b) 
all designations which would be 
permitted under the PCT except any 
designation(s) of the State(s) indicated 
under item V-6 below. The applicant 
declares that those additional 
designations are subject to confirmation 
and that any designation which is not 
confirmed before the expiration of 15 
months from the priority date is to be 
regarded as withdrawn by the applicant 
at the expiration of that time limit. 
Exclusion(s) from precautionary 

designations 

Priority claim of earlier national 
application 
Filing date 

VI-1-2 [Number 

Vl-1-3 Country 



NONE 



VI -2 I Priority document request 

The receiving Office is requested to 
prepare and transmit to the 
International Bureau a certified copy of 
the earlier application(s) identified 

above as item(s): 

VlM I International Searching Authority 
Chosen 



04 March 2002 (04 

20020414 

FI 

VI-1 



03 .2002) 



Swedish 



Office (ISA/SE) 



VII-2 | Request to use results of earlier 
search; reference to that search 

VII-2-1 [Date 
VII-2-2 Number 

Vll-2-3 Country (or regional Office) 



VIII 

Vlll-1 

VIII-2 



12 December 
20020414 
FI 



2002 (12.12.2002) 



VIII-3 



VIII-4 



VIH-5 



Declarations 

Declaration as to the identity of the 

inventor 

Declaration as to the applicant's 
entitlement, as at the international filing 
date, to apply for and be granted a 
patent 



Number of declarations 



Declaration as to the applicant's 
entitlement, as at the international filing 
date, to claim the priority of the earlier 

application 

Declaration of inventorship (only for the 
purposes of the designation of the 
United States of America) 



Declaration as to non-prejudicial 
disclosures or exceptions to lack of 
novelty " 



i 



■0 ' 
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vin-2-1 



Vlll-2-1 
(iv) 

Vlll-2-1 



Declaration: Entitlement to apply for 
and be granted a patent 

Declaration as to the applicant's 
entitlement, as at the international filing 
date, to apply for and be granted a 
patent (Rules 4.17(H) and 51bis.1(a)(ii)), 
in a case where the declaration under 
Rule 4.17(iv) is not appropriate: 
Name: 



This declaration is made for the 
purposes of: 



in 



relation to this 



application 



NOKIA CORPORATION 

is entitled to apply for and be granted 
a patent by virtue of the following: 
an assignment from FLANAGAN, Adrian to 
NOKIA CORPORATION, dated 22 March 2002 

(22.03.2002) 

all designations except the designation 
of the United States of America 
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